In hybrid video coding frameworks, filtering techniques are typically used for removing compression artifacts or for anti-aliasing. Such filtering techniques can be used in-loop or out-of-loop in many video compression applications. Due to the non-stationary nature of video signals, the filtering processes are expected to be adaptive in both spatial and temporal domains. Recently, many adaptive filtering schemes have been proposed to target and better restore coded pictures in order to improve coding efficiency.
Video coding standards employ block-based transforms (e.g., discrete cosine transforms (DCTs)) and motion compensation to achieve compression efficiency. Due to the lossy-compression property of the framework, the quality of the reconstructed videos is degraded. In the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) Standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 Recommendation (hereinafter the “MPEG-4 AVC Standard”), a deblocking filter is applied to the decoded picture for removing blocky artifacts. In the Key Technology Area (KTA) of the MPEG-4 AVC Standard, Wiener filters are used in-loop or out-of-loop to improve the quality of decoded pictures. To achieve temporal adaptation, a Wiener filter is usually estimated for each picture by minimizing the mean square error between the original picture and the decoded picture at the encoder. The filter coefficients are sent to the decoder (in the bitstream or on the conveying media) as overhead for each picture. To achieve spatial adaptation, a picture can be divided into many regions. In each region, a switch controls whether the filtering process is “on” or “off”. The region partition information and a switch control message are sent to the decoder as side information. However, the cost for sending this side information is high and not affordable in some cases, especially in many low bitrate but high resolution applications.
Furthermore, without any constraint on the data set for filter estimation, there are often outliers in the data set used in the estimation of the filter. The Wiener filter estimated by minimizing the mean square error of the data set is usually sub-optimal (or far from optimal).
Post Filter in KTA
In the Key Technology Area (KTA) of the MPEG-4 AVC Standard, an adaptive post filter was proposed in a first prior art approach. The basic idea of the first prior art approach is to apply a Wiener filter to the decoded picture before display. The Wiener filter can be estimated for each picture by minimizing the mean square error (MSE) between the original picture and the decoded (i.e., reconstructed) picture at the encoder. In this case, the estimated filter coefficients are sent to the decoder as overhead. The whole picture is filtered with the estimated filter. In another case, a set of Wiener filters are trained offline, and transmitted to or stored at the decoder. During decoding, the picture is filtered pixel-wise. At each pixel, a filter is selected from the filter set based on the statistics of the surrounding pixels. In this technique, the filtering indicator does not cost any overhead. The filtering indicator can be derived by the decoded picture content. However, the model between the filtering indicator and the picture content is very difficult to build. In the first prior art approach, an offline training scheme is exploited to find the corresponding model between the filtering indicator and the local variance. Since training is highly dependent on the training set, the model trained with limited data will be inaccurate for more general video signals.
Adaptive Loop Filter in KTA
An adaptive loop filter was proposed in a second prior art approach. Similar to the post filtering performed with respect to the first prior art approach described above, the Wiener filter is estimated for each picture by minimizing the mean square error (MSE) between the original picture and the decoded picture at the encoder. Since the Wiener filter is optimal in the sense of the mean square error of the whole picture, the quality of some regions can be degraded when applying the Wiener filter. To remove these pixels during filtering, a region based solution is proposed to achieve a trade-off between the overhead cost and coding performance. The second prior art approach indicates the regions where the Wiener filter is applied. The filter indicator is sent to the decoder as side information. By allowing different regions to have different block-sizes, a quad-tree-based flag can save bits by sending fewer bits in a large block-size region. All these methods require a block-based binary map to cover the entire frame.